Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add TLS configuration for RPK connecting to the schema registry #1592

Merged
merged 1 commit into from
Nov 8, 2024
Merged

Add TLS configuration for RPK connecting to the schema registry #1592

merged 1 commit into from
Nov 8, 2024

Conversation

andrewstucki
Copy link
Contributor

Currently installing a cluster with TLS enabled break RPKs interactions with the schema registry:

$ kubectl exec -it -n redpanda basic-0 -c redpanda -- rpk registry schema list
unable to GET "http://127.0.0.1:8081/subjects": Get "http://127.0.0.1:8081/subjects": EOF
command terminated with exit code 1

Where you can access it properly with curl:

$ kubectl exec -it -n redpanda basic-0 -c redpanda -- curl https://basic-0.basic.redpanda.svc.cluster.local:8081/subjects -sS --cacert /etc/tls/certs/default/ca.crt -sS -w '\n'
[]

This is because we're missing a schema_registry stanza from the rpk node configuration block:

rpk:
    kafka_api:
        brokers:
            - basic-0.basic.redpanda.svc.cluster.local.:9093
        tls:
            ca_file: /etc/tls/certs/default/ca.crt
    admin_api:
        addresses:
            - basic-0.basic.redpanda.svc.cluster.local.:9644
        tls:
            ca_file: /etc/tls/certs/default/ca.crt
    additional_start_flags:
        - --default-log-level=info
        - --memory=2048M
        - --reserve-memory=205M
        - --smp=1
    tune_aio_events: true

This adds the proper configuration so that RPK can interact with the schema registry.

david-yu
david-yu approved these changes Nov 5, 2024
View reviewed changes
Copy link
Contributor

@david-yu david-yu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@andrewstucki
Copy link
Contributor Author

Holding off on merging for a bit, realized that the schema_registry part of redpanda.yaml's rpk stanza isn't actually implemented and currently connection parameters for the schema registry can only be set via rpk profiles. I have a fix to bring it into the rpk stanza in repanda.yaml as well:

https://github.com/redpanda-data/redpanda/compare/allow-setting-schema-registry-via-rpk-node-config

But waiting to hear back on how we want to proceed, either just plugging the above for now or reworking the chart to use profiles, which sounds like it's likely the "preferred" way we tell folks to use RPK.

@andrewstucki
Copy link
Contributor Author

Opened up redpanda-data/redpanda#24017 to implement being able to configure this in redpanda.yaml.

@david-yu
Copy link
Contributor

david-yu commented Nov 7, 2024

I assume we need to wait for an rpk build (i.e. 24.3) to be out for this to merge? Or would it not matter? I assume docs with the rpk command would need to wait though for 24.3.

@andrewstucki
Copy link
Contributor Author

@david-yu I'm going to go ahead and merge it -- it'll do nothing for the time being since older versions of rpk won't understand the stanza that was added and will just ignore it. I'll re-validate it once we cut the rpk/redpanda release.

@andrewstucki andrewstucki merged commit c7c21fe into main Nov 8, 2024
43 checks passed
@andrewstucki andrewstucki deleted the fix-schema-registry-rpk-stanza branch November 8, 2024 19:05
@david-yu
Copy link
Contributor

@andrewstucki I assume its possible to test with this RC release? https://github.com/redpanda-data/redpanda/releases/tag/untagged-5fe25a9a58fdcd612efd

@BenPope
Copy link
Member

BenPope commented Nov 14, 2024

https://hub.docker.com/r/redpandadata/redpanda-unstable/tags?name=v24.3.1-rc4

@andrewstucki
Copy link
Contributor Author

Thanks @BenPope!

@david-yu - works like a charm.

➜ kubectl exec -it -n redpanda basic-0 --container redpanda -- rpk registry schema list
SUBJECT        VERSION  ID    TYPE  ERROR
mytopic-value  1        1     AVRO

RafalKorepta pushed a commit to redpanda-data/redpanda-operator that referenced this pull request Dec 3, 2024
@andrewstucki
Add TLS configuration for RPK connecting to the schema registry (redp…
abdacd9 
…anda-data/helm-charts#1592)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chrisseto chrisseto chrisseto approved these changes

@david-yu david-yu david-yu approved these changes

@RafalKorepta RafalKorepta Awaiting requested review from RafalKorepta RafalKorepta is a code owner

@JakeSCahill JakeSCahill Awaiting requested review from JakeSCahill

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andrewstucki @david-yu @BenPope @chrisseto